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ABSTRACT 



This study evaluates the effect of treatment components 
through a secondary analysis of data from -the National Treatment Improvement 
Evaluation Study (NTIES) . The study examines the relationship between 
treatment components, client- level factors, and treatment outcomes, and how 
these relationships vary by treatment modality. It seeks to understand the 
response by NTIES clients to components of drug and alcohol treatment defined 
in ways that have external applicability and that are immediately available 
to care providers. Several dose components were shown to have significant 
effects on the odds of positive outcome. Several interactions between 
client -level factors and dose components were observed. Compared to the 
background variables alone, the inclusion of treatment components 
significantly improved the logistic regression models for non-methadone 
outpatient and short-term residential modalities both for outcomes and for 
methadone for treatment substance abstinence. Improvement for the long-term 
residential modality was marginal and not significant for the correctional 
modality. Implications for research, policy, and practice are discussed. 
Appendix A is "Description of the National Treatment Improvement Evaluation 
Study and Center for Substance Abuse Treatment Demonstrations (1990-1992)"; 
Appendix B is "Distributions of SDU Means for Treatment Components within 
Modalities"; and Appendix C is "Technical Notes." (Contains 10 figures, 8 
tables, and 21 references.) (MKA) 
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Foreword 



The Center for Substance Abuse Treatment (CSAT) works to improve the lives of those 
affected by alcohol and other substance abuse, and, through treatment, to reduce the ill effects of 
substance abuse on individuals, families, communities, and society at large. Thus, one important 
mission of CSAT is to expand the knowledge about and the availability of effective substance 
abuse treatment and recovery services. To aid in accomplishing that mission, CSAT continues to 
invest significant resources in the development and acquisition of high-quality data about 
substance abuse treatment services, clients, and outcomes. 

In support of these efforts, the Program Evaluation Branch (PEB) of CSAT established 
the National Evaluation Data Services (NEDS) contract to provide a wide array of data 
management and scientific support services across various programmatic and evaluation 
activities and to mine existing data whose potential has not been fully explored. Essentially, 
NEDS is a pioneering in that the CSAT previously had no mechanism established to pull 
together databases for broad analytic purposes or to house databases produced under a wide arra 
of activities. One of the specific objectives of the NEDS project is to provide CSAT with a 
flexible analytic capability to use existing data to address policy-relevant questions about 
substance abuse treatment. This report has been produced in pursuit of that objective. 

How drug and alcohol abuse treatment outcomes are affected by different amounts and 
components of treatment is fundamentally important to evaluating current treatment practices and 
improving treatment in the future. The present study operationalized and evaluated components 
of treatment through a secondary analysis of data from the National Treatment Improvement 
Evaluation Study (NTIES) (see Appendix A) and used multivariate analyses to model the effect 
of the components on two dichotomous outcome measures: abstinence at follow-up of a 
substance for which the client sought treatment and abstinence at follow-up of all substances, 
including alcohol. The study examined the relationships among treatment components, client- 
level factors, and treatment outcomes and how these relationships vary by treatment modality. 

The results presented here suggest that greater intensity of treatment can improve 
outcomes in short-term residential settings and that treatment plans can be useful in man 
settings. They also suggest that longer lengths of stay may enhance treatment outcomes in non- 
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methadone outpatient and long-term residential modalities, even when patients do not complete 
treatment, and that research into related services such as aftercare and overutilization could be 
productive. 

Sharon Bishop 
Project Director 

National Evaluation Data Services 
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Executive Summary 



1. INTRODUCTION 

How different amounts and components of treatment affect drug and alcohol abuse 
treatment outcomes is fundamentally important to evaluating current treatment practices and 
improving treatment in the future. Additionally, understanding the interaction between client 
severity, client characteristics, and treatment components can point to more effective treatment 
strategies. However, understanding relationships between treatment components and outcomes 
is complicated by a number of factors: 1) different populations respond to different treatment 
components, 2) it is difficult to tease out the effects of specific components in a comprehensive 
treatment environment without experimental designs that isolate those components, and 
correspondingly, 3) it is difficult to control for client self-selection into treatment services. Yet 
the relationship between treatment components and outcomes is something that the treatment 
field must understand better. The downstream implications for the treatment communities 
(e.g., practice guidelines), the behavioral health care industry (e.g., managed care organization 
coverage of substance abuse treatment), and of course the clients (more cost-effective treatments 
and increased quality of life) are enormous. 

The present study evaluated the effect of treatment components through a secondar 
analysis of data from the National Treatment Improvement Evaluation Study (NTIES). The 
study examined the relationships between treatment components, client-level factors, and 
treatment components and how these relationships vary by treatment modality. It sought to 
understand the response by NTIES clients to components of drug and alcohol treatment defined 
in ways that have external applicability and that are immediately available to care providers. 

2. METHODOLOGY 

NTIES data for 3,085 clients across five treatment modalities were used in this study. 

The five modalities were methadone maintenance, non-methadone outpatient, short-term 
residential, long-term residential, and correctional. All analyses were conducted separately b 
modality so as not to obscure meaningful differences in effects of treatment components in 
different modalities. 

Treatment components included in the analysis were: length of stay in treatment; hours 
spent in counseling sessions per month; general measures of drug and alcohol, medical, and 
mental health treatment; ancillary and supportive services; client-treatment matching (whether 
the SDU reported it matched most or all of its patients); prominence of treatment plans (whether 
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the patient said they saw “a treatment plan or a list of treatment goals”); average weekl 
methadone dose (methadone modality only); and certain other medications. A number of client- 
level factors were included as background variables to partially control for self-selection and 
permit assessment of interactions between client characteristics and treatment components. 



A multilevel mixed-effects logistic regression model was used to estimate the effect of 
treatment components on two measures of abstinence: abstinent at follow-up from a substance 
for which the client sought treatment and abstinent from all substances. Estimates of the overall 
contribution of treatment components in explaining outcomes (over and above background 
variables) were also examined. 

3. RESULTS 

Several dose components were shown to have significant effects on the odds of a positive 
outcome; 



■ Length of stay showed a consistently positive effect on both treatment substance and 
all substance abstinence; however, the effects were statistically significant in the non- 
methadone outpatient and long-term residential modalities only. In these modalities, 
the odds ratios suggest that controlling for other variables and increasing length of 
stay by 30 days can improve the odds of a successful outcome by as much as 15 
percent. 

■ Whether or not clients reported seeing their treatment plan also was consistent! 
positive across all modalities, though statistically significant in only the methadone 
and short-term residential modalities, where seeing treatment plans increased the odds 
of all substance abstinence by 200 and 138 percent, respectively. 

■ Asa main effect, hours per month in group and individual counseling had a 
significant positive effect in the short-term residential modality only. Educational, 
vocational and other ancillary services, the use of antianxiety and drug and alcohol 
medications, and patient matching also had positive impacts on one or both forms of 
abstinence in short-term residential Service Delivery Units (SDUs), suggesting that 
greater treatment intensity may pay off in that modality. 

■ One difference between the two endpoints is the significance and strength of social 
and family services in non-methadone outpatient and correctional facilities 
modalities. This component is significant and positive in all-substance abstinence but 
not in treatment substance abstinence. 
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Several interactions between client-level factors and dose components were observed, including 
the following: 

■ Significant interactions were observed between hours in group or individual 
counseling with the percent of needed social/family services received and the percent 
of needed employment/educational services received. Generally, the interactions with 
these two indices tended to attenuate the effects of hours per month in group or 
individual counseling, in that the interaction terms are positive when the effect of 
hours per month is negative (methadone, long-term residential) and negative when 
hours per month the effect of hours per month is positive (short-term residential). 

■ While women were significantly more likely to be abstinent in most modalities, the 
positive effects of hours per month in counseling, substance and alcohol medications, 
percent of needed services received, and treatment plans appeared to differential! 
benefit men. This suggests that providers may need to offer more tailored approaches 
to these services so that women benefit as well. 

■ Treatment completion interacts with length of stay (non-methadone outpatient) and 
AA/NA/CA attendance during treatment (non-methadone outpatient and short-term 
residential). In the case of meeting attendance in the non-methadone outpatient 
modality, the main effect was not significant while the interaction was positive, 
suggesting that in this modality attendance at meetings may be more beneficial among 
clients who complete treatment. This may reflect the fact that meeting attendance and 
treatment completion are both manifestations of a single, unmeasured factor such as 
motivation, which in turn may be most likely to affect outcomes in the relativel 
unstructured environment of outpatient treatment. 

Compared to the background variables alone, the inclusion of treatment components significant! 
improved the logistic regression models for non-methadone outpatient and short-term residential 
modalities both for outcomes and for Methadone for treatment substance abstinence. Improvement 
for the long-term residential modality was marginal p=.054 for treatment substance abstinent and 
p=.095 for all substance abstinent) and not significant for the correctional modality. Using the 
background models as the base, adding the treatment components increased the estimated variance 
accounted for by 1-2 percent in correctional modalities and 3-8 percent in the other modalities. 

The following limitations should be considered in interpreting these results: 

■ Components of dose in some instances are not always definitive. Some questionnaire 
items are ambiguous, and refining the components will require continued work. 

■ Inferences from non-experimental data are inherently vulnerable to selection bias. As 
described in the report, the hierarchical set approach in which variance common to 
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background factors and dose components is “credited” to background factors offers 
some measure of protection against the misinterpretation of selection bias as 
treatment effects, but only to the extent that the selection bias is captured by the 
background factors that were measured and included in the models. 

■ Perhaps the most significant limitation is the inability to place the SDU episode in the 
context of the client’s treatment continuum and the corresponding implications for 
interpreting dose. For example, because questions about services received were 
usually restricted to the dates of NTIES entry and treatment completion, there was no 
way to obtain information about phases of treatment obtained just before entering 
NTEES, and information about phases of treatment within the NTEES episode was not 
consistently reported. 

4. IMPLICATIONS FOR RESEARCH, POLICY, AND PRACTICE 

The following is a list of research areas that could productively be investigated with 

NTEES data: 

■ The effect of the aftercare component through the use of the follow-up instrument to 
investigate and quantify post-treatment services 

■ The effect of different urinalysis regimens during treatment 

■ The impact of assessment at intake, with emphasis on the interplay between 
assessment activities at intake and the use of treatment plans and client matching 

■ The extent and impact of clients receiving services they did not need and would not 
benefit from 

■ Whether alternate conceptualizations of treatment components yielded similar results; 
such a finding would increase confidence in the results reported here. 

Implications for policy and practice that can be drawn from these results are: 

■ The positive relationships with hours per month in group and individual counseling, 
the percentage of needed educational and vocational counseling received, and 
substance and alcohol medications suggest that greater treatment intensity in short- 
term residential SDUs can improve treatment outcomes. 

■ Longer lengths of stay may enhance treatment outcomes in non-methadone outpatient 
and long-term residential modalities, even when patients do not complete treatment. 
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■ Clients who see and participate in the development of their treatment plans can 
potentially enhance treatment outcomes with little cost or risk. We found little 
evidence that it was generally helpful or that it diverted resources from other 
treatment strategies, and little evidence that it was generally helpful in all modalities. 

■ Matching clients to counselors or key providers appears to improve treatment 
outcomes in methadone maintenance and correctional modalities but not in long-term 
residential facilities. 

■ The apparent effectiveness of treatment components like counseling hours per month, 
employment and vocational services, and substance and alcohol medications var 
considerably by modality. This suggests that modality should be taken into account 
before recommending implementation or expansion of these components. 
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I. Introduction 



1. STATEMENT OF THE PROBLEM 

How different amounts and components of treatment affect substance and alcohol abuse 
treatment outcomes is fundamentally important to evaluating current treatment practices and 
improving treatment in the future. Additionally, understanding the interaction between client 
severity, client characteristics, and treatment components can point to more effective treatment 
strategies. However, understanding relationships between treatment components and outcomes 
is complicated by a number of factors: 1) different populations respond to different treatment 
components, 2) it is difficult to tease out the effects of specific components in a comprehensive 
treatment environment without experimental designs that isolate those components, and 
correspondingly, 3) it is difficult to control for client self-selection into treatment services.' Yet 
the relationship between treatment components and outcomes is something that the treatment 
field must understand better. The downstream implications for the treatment communities 
(e.g., practice guidelines), the behavioral health care industry (e.g., managed care organization 
coverage of substance abuse treatment), and of course the clients (more cost-effective treatments 
and increased quality of life) are enormous. 

2. THE PRESENT STUDY 

The National Treatment Improvement Evaluation Study (NTIES) provides an excellent 
vehicle for examining these issues in naturalistic treatment settings. The NTIES surve 
(described in Appendix A) was a 5-year longitudinal study of the impact of substance abuse 
treatment on a total of 5,388 clients purposively sampled from public substance abuse treatment 
programs (service delivery units or SDUs) that were funded by the Center for Substance Abuse 
Treatment (CSAT). Data were collected from five SDU modalities: methadone maintenance, 
non-methadone outpatient, short-term residential, long-term residential, and correctional. 

The present study uses a multilevel random-effects analysis to model the effect of 
treatment components on dichotomous outcome measures — abstinent at follow-up from a 



' For example, it is possible to show no relationship, or even inverse relationships, between intensity of services and 
outcomes (Orwin et al., 1994). The “worse-off’ clients can require the most services, yet because they are worse 
off initially, their prognosis is poorer. Hence the need for case-mix adjustment as a means of controlling for client 
severity. 
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substance for which the client sought treatment and abstinent from all substances? Components 
were drawn from items in the data set that pertain to prominent issues in substance and alcohol 
treatment literature. The primary goal of this analysis was to evaluate measures that are 
meaningful and usable in the field, not just to identify a closely related set of variables in the 
NTIES data set. It sought to understand the response by NTIES clients to components of 
substance and alcohol treatment defined in ways that have external applicability and that are 
immediately available to care providers. 

3. STUDY QUESTIONS 

The present study addresses the following specific questions: 

■ What treatment components appear to improve outcomes, and how do these 
relationships vary by modality? 

■ What interactions occur between client-level factors and treatment components 

■ How much do the treatment components improve prediction of good outcomes 

■ What are the implications for future research, policy, and practice 



^ Operational definitions of abstinence and other terms are described in Chapter I, Methods. General terms 
regarding NTIES are defined in Appendix A. 
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II. Methods 



This section describes the sample, outcome measures, background and treatment 
components, and the statistical techniques used in the analysis. 

1. DESCRIPTION OF THE SAMPLE FOR ANALYSIS 

The NTIES data set contained 4,41 1 clients with intake and follow-up data with treatment 
outcomes. However, only 3,681 of this number also had exit questionnaires, and only 61 of the 
71 SDUs from which the 4,41 1 clients were drawn had Baseline Administrative Forms (NEAR), 
a self-administered questionnaire typically completed by the SDU director. 

This analysis drew from all sources within the NTIES data set, and as a result was based 
on a subset of 3,085 clients with follow-up for whom all the necessary information was available. 
A total of 3,085 clients were used, after excluding 451 clients from SDUs with no NEAR 
information, 730 clients without an exit questionnaire, and 145 clients with unknown information 
on the variables included in final models. Generally, variables with 30 percent missing data were 
excluded from analysis, but this was not always possible when several variables describing 
relatively rare events and presenting ambiguous missing codes were combined for analysis (see 
Appendix C for additional details). 

2. OUTCOME MEASURES 

Two outcome measures were defined: “Treatment Substance Abstinent” defined a 
positive outcome as being abstinent at follow-up from any of the substances for which the client 
presented, while “All Substance Abstinent,” was more restrictive, requiring the client to be 
abstinent from all substances at follow-up. By “substance,” we included all substances plus 
alcohol (but not tobacco). So, for example, treatment substance abstinence would treat a client 
who gave up heroin for cocaine as a positive outcome, despite research indicating that addicts 
may try to modulate mood with cocaine or other substances when they can no longer do so with 
heroin (CSAT, 1993). In the NTIES data, however, substitution of a more serious substance was 
rare, and substitution of a comparable substance relatively uncommon.^ In most cases, therefore. 



^ Ten clients who presented for crack reported using heroin at follow-up but not crack, and 4 who presented for 
alcohol or marijuana reported using crack, cocaine, or other substances at follow-up but not alcohol or marijuana. 
Additionally, 53 clients reported using crack after treatment for cocaine, and 15 used cocaine after treatment for 
crack. 
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treatment substance abstinence represented harm reduction. For both outcome variables, as 
specified in the original survey (NORC, 1997), “abstinent” was defined as trying the substance 
fewer than five times during the year-long follow-up period. 

Of the 3,085 clients used in this analysis, 1,41 1 (46%) were “Treatment Substance 
Abstinent” and 972 (32%) were “All Substance Abstinent.” Exhibit II- 1 presents treatment 
outcomes by modality. 



’Si, . ;■ i--. XREATlVffiNT'OUTCOMES"BYMopALITY| , 




METHADONE 
S (n=295) S; ■ 


^METHADONE! 

'OtTPATIENTS 


SHORT^TER% 
RESIDENTIAL 
■ (n=678) ^ S 


SiLONG^TERM i 
RESIDENTIAL 


CORRECTIONAL 
FACILITIES ^ 

'Tn=610) ' 


TOTALS 

(n:i3i085) 


Treatment 

Substance 

Abstinent 


90 (31%) 


432 (47%) 


288 (42%) 


283 (49%) 


318(52%) 


1,411 

(46%) 


All 

Substance 

Abstinent 


47 (16%) 


328 (35%) 


189 (28%) 


214(37%) 


194 (32%) 


972 

(32%) 



3. BACKGROUND VARIABLES 

Treatments may vary in their applicability to different subgroups. Generally, analyses can 
treat these groups as special cases and analyze them separately, but this can greatly reduce the 
statistical power of the models as cells are repeatedly subdivided. Preliminary modeling indicated 
that stratification beyond the five modalities would adversely affect statistical power. As a result, 
age, gender and pregnancy status, education, race, and ethnicity were included in modeling as 
background variables and are listed in Exhibit II-2. Additionally, clients’ primary substance, 
treatment completion status, prior treatment, medical and mental conditions, and legal pressures 
were also included as background variables, consistent with the original analysis (NORC, 1997). 
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BACKGROUND VAIOABLES USED m LOGISTIC REGRESSION MODELS 


BACKGROUND VARIABLES ' 


Description 


Data Source(s): 


Age 


Race/Ethnicit 


Race/Eihnicii 


Race (R44 - NRIQ) 


Hispanic 


Hispanic Origin (R43 - NRIQ) 
Languages Spoken (R47 - NRIQ) 
Language Preferred (R50 - NRIQ) 


Gender 


Gender (R29 - NRIQ) 


Pregnancy Status 


Pregnant at Intake (R470) 


Years of Education 


Highest Grade Attended (R58 - NRIQ) 


Prior Treatment Episode 


Prior Alcohol Treatment (R255 - NRIQ) 
Prior Drug Treatment (R256 - NRIQ) 


Treatment Completion 


Analysis file (TXCOMPL) 


Legal Pressure to Seek Treatment 


Drug Treatment Recommended by CJS (R373 - NRIQ) 


Comorbid Medical Conditions 


Medical Conditions (R427 - NRIQ) 

Client Diagnosed with AIDS (R428 - NRIQ) 

Client Has Other Health Problems (R429 - NRIQ) 

No. of Nights in Hospital (R454A1 - NRIQ) 

No. of Visits to ER (R454A2 - NRIQ) 

No. of Visits to Doctor’s Office/Clinic (R454A3 - NRIQ) 
Was Drug/Alcohol Reason for Visits (R456A1-A3 - NRIQ) 


Comorbid Mental Health Conditions 


Panic Disorder (R289 & R294 - NRIQ) 
Depression (R297 & R298 - NRIQ) 

Suicidal Tendencies (R302, R303, R306, R307) 
Hallucinations (R309) 

ADDH (R313A1-A3) 


In Treatment for Heroin 


(R113M9-NRIQ) 


In Treatment for Cocaine 


(R113M4-NRIQ) 


In Treatment for Alcohol 


(R113M12-NRIQ) 
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4. TREATMENT COMPONENTS 

The following describes each component of treatment used in the model. 

4.1 Length of Stay in Treatment 

Total length of stay in treatment has shown a positive relationship with post-treatment 
outcomes (De Leon, 1991; Etheridge, Craddock, Dunteman, & Hubbard, 1995; Hser, Anglin, & 
Liu, 1991; McKusker, Stoddard, Frost, & Zorn 1996; Tims, Horton, Fletcher, & Price, 1992). 

This component was defined as the length (in days) of the patient’s treatment episode. 

4.2 Individual and Group Counseling (Hours Per Month) 

Total time spent in individual, group, and family counseling, and total time spent in 
classrooms was investigated as a treatment component. Numerous studies discuss the effect of 
counseling on outcome (e.g., Brochu, Landry, Bergeron, & Chiocchio, 1997; Daley, Salloum, 
Zuckoff, Kinisci, & Thase, 1998; McLellan et al., 1994; Ouimette, Ahrens, Moos, and Finney, 
1998), and it is an aspect of treatment that is relatively easy to quantify. Total time normall 
spent in counseling sessions is reported by treatment staff in administrative questionnaires. Time 
spent in counseling is reported for individual clients by staff members in a patient abstract 
questionnaire and by clients themselves in exit questionnaires. To include the largest number of 
clients in the analysis, a variable was constructed using all available information, using first the 
self-reported data, then the data reported about the client by staff, and then the administrative 
information. 

4.3 General Measures of Substance and Alcohol, Medical, and Mental Health Treatment 

Two variables depicting general substance and alcohol treatment were defined from the 
client-reported exit questionnaire. These were: 1) whether clients reported going to Cocaine 
Anonymous (CA), Narcotics Anonymous (NA), or Alcoholics Anonymous (AA) during 
treatment, and 2) whether clients reported they took prescribed substance and alcohol 
medications during treatment. 

Three other variables identified clients who received HIV and TB testing, necessar 
medical care, and mental health care. Clients were deemed to have received treatment for 
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medical conditions if they reported that they received any kind of medical services.'* Treatment 
for mental health conditions was based upon the client’s self report at exit of “counseling or 
treatment for problems with your emotions, nerves, or mental health.” 

4.4 Ancillary and Supportive Services 

The amount and quality of treatment services has been shown to predict outcomes (APA, 
1995). In addition to group and individual counseling, services focusing on interpersonal skills, 
family counseling, child care, and parenting skills are provided in all modalities depending upon 
client need. Two indices were constructed to handle important ancillary services that SDUs offer 
but that may not always be needed by all clients. Generally, whether a client received a service 
was taken from self-reported information in the exit questionnaire, and whether the service was 
needed by the client was determined using the client’s stated priorities at intake. The first index 
was defined to address social and family needs: family counseling, interpersonal skills 
counseling, parenting counseling, and child care. A proportion was derived using as the 
denominator the number of four kinds of family services that had been met or were reportedl 
needed at intake. The numerator was defined as the actual number of services received. The 
four kinds of family services are as follows. Clients were asked at exit if they received: 

■ Individual, group, or family counseling for family problems (it was deemed needed if 
at index he/she stated that counseling for abuse or help with other family problems 
was very important to him/her) 

■ Help with interpersonal skills (it was deemed needed if he/she stated at intake that 
help getting along with people was very important to him/her) 

■ Counseling or classes for child rearing (it was deemed needed if at intake he/she 
stated that training about raising children is very important to him/her) 

■ Child care or money for child care (it was deemed needed if at intake h/she stated 
he/she has difficulty getting child care). 

For example, if a client reported at intake that child care and interpersonal skills were ver 
important, but received only child care services, an index of 0.5 or 50 percent was calculated. If 
the same client also received classes for child care, child care would be added to both the 



■* The condition or injury that required medical attention was not identified, but clients with pre-existing medical 
conditions were identified with one of the background variables. 
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numerator and the denominator for an index of 66 percent, since it was assumed that services 
received were needed.^ The second index is based upon vocational, educational, legal, housing 
and transportation services, and assistance in securing financial benefits. It was constructed in 
the same fashion from the following six elements. Clients were asked at exit if they received: 



■ Help with educational services (it was deemed needed if at intake the client had less 
than a high school education) 

■ Help with employment skills (it was deemed needed if at intake the client stated that 
employment counseling was very important to him/her) 

■ Help with housing (it was deemed needed if at intake the client stated that help with 
housing was very important to him/her) 

■ Help with legal problems (it was deemed important if at intake the client was on 
probation, parole, work release, or had been recently released) 

■ Help to get benefits from the government (it was deemed needed if at intake the client 
stated that help financial problems was very important to him/her) 

■ Help with transportation (it was deemed needed if at intake the client stated that 
he/she thought transportation problems might make it hard for him/her to get 
treatment). 

4.5 Matching 

Matching has been described as the process of individualizing treatment resources to 
patient needs and preferences (CSAT, 1995). It has also been described as a placement criteria 
for selecting which treatment setting and intensity is appropriate, as described in the American 
Society of Addiction Medicine placement criteria (CSAT, 1994). Within the NTIES data set, 
client-treatment matching is addressed only in the administrative questionnaire, in which staff 
members described their program. Moreover, the questionnaire only asks whether the program 
matches all, most, or some clients, and it does not distinguish which clients are matched. For this 
analysis, a client was deemed to have benefitted from the practice of matching if the SDU 
reported that it matched all or most clients. 



’ Potential problems with this assumption are addressed in the limitations section. 
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4.6 Treatment Plans 

A formalized treatment plan is required by Federal methadone regulations, the Joint 
Commission on the Accreditation of Healthcare Organizations (JCAHO), and the Commission 
on the Accreditation of Rehabilitation Facilities (CARF) (CSAT, 1994). 

Clients were asked whether they saw “a treatment plan or a list of treatment goals” in the 
exit questionnaire. It describes a treatment plan as something that “can include things such as 
getting off substances, not using alcohol, improving your health, or getting a job,” and does not 
present it as a formally developed, written document. However, it serves as an indication of the 
priority placed on treatment planning throughout the client’s care and was included as a 
dichotomous treatment component.® 

4.7 Average Weekly Methadone 

Average weekly methadone dosage was used as a treatment component for the methadone 
modality only. Methadone dosage is recorded in the NTIES data set with date and daily dose in 
milligrams. The manner in which the information was filled out depended upon the length of 
time the client was in treatment, so that a client’s dose might be recorded every day, 
week, or month. Average dosage at any given point in time was extrapolated from these 
“benchmarks.”’ 

4.8 Other Medications 

Two general categories of substances — analgesics and antianxiety medications — were 
defined dichotomous treatment components based upon the staff-reported information in the 
client abstract.^ 



Questions about treatment plans also appear in the administrative questionnaire, however their wording makes it 
difficult to distinguish nonresponses from responses intended to be negative, and whether or not informal 
treatment planning is included. 

’ Methadone maintenance often begins with a dose adjustment. The “State Methadone Treatment Guidelines” 
(CSAT, 1993) states that “an initial (methadone) dose of 20-30 mg is usually safe” to relieve withdrawal. Dose 
changes to reach the tolerance threshold were ignored in favor of the level reached for long-term maintenance, 
which was assumed to begin in the fourth week of treatment. 

* The use of other prescribed medications was too incompletely reported to be used. 
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5. STATISTICAL ANALYSIS 

5.1 Descriptions of Treatment Component Distributions 

Descriptive statistics were computed on all treatment components by modality and b 
SDU within modality. Modality means were tabled, and the distributions of SDU means b 
modality were plotted and described. 

5.2 Logistic Regression Models 

The impact of treatment components was evaluated using logistic regression. Adjusted 

odds ratios (P/l-P) received treatmenl componeni / (P/1~P) did not receive ireaimem componeni Were Calculated b 

exponentiating each coefficient from the final model. Model fit and comparisons between 
models with and without treatment components were evaluated with two “goodness of fit” 
measures. Model improvement was evaluated using the difference in the -2 log likelihood of 
each model, and estimates of the total variance accounted for by each model before and after 
treatment components also were examined (Hosmer & Lemeshaw, 1989). 

To account for clustering within each service provider, logistic regression modeling used 
restricted maximum likelihood estimation and the SAS macro GLINMIX. Within each separate 
model for each modality, SDU was included as a random error term, and treatment components 
measured at the individual level were treated as fixed effects. 

First-order interactions among all treatment components and between treatment 
components and selected background variables (age, gender, race/ethnicity, and treatment 
completion) were examined one at a time and initially included in the models if their p value was 
0. 10 or less. Some interaction terms no longer met the p<0. 10 criteria after other interaction 
terms were included and were removed. Since no specific interactions were predicted a priori, 
and the examination of a large number of possible interactions potentially capitalizes on chance, 
these analyses should be viewed as strictly exploratory. 
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This section presents the results of the study analyses. The sections include distributions 
of the treatment components and their contribution to the logistic regression models. 

1. DISTRIBUTIONS OF TREATMENT COMPONENTS 

Exhibit III-l presents, for each modality, the means of continuous treatment components 
such as length of stay and hours per month of counseling, means of percent needed services 
received, and percentage of clients meeting criteria of categorical treatment components. The 
distribution of SDU means within each modality for the various components appear in Appendix B. 

As expected, mean length of stay was highest in the methadone SDUs, where clients 
could often remain in maintenance indefinitely. Non-methadone outpatient and long-term 
residential were the next longest (127 and 107 days, respectively). Short-term residential and 
correctional facilities were the shortest (50 and 70 days, respectively) and also had the least 
Variability across SDUs (see Appendix B, Exhibit B-1). On the other hand, short-term residential 
and correctional facilities clients had the highest mean hours per month of counseling with their 
principal provider (17.0 and 17.6, respectively) as well as the most variability across SDUs (see 
Appendix B, Exhibit B-2), while methadone clients had the lowest means (5.1) with little 
variability across SDUs. In addition, short-term residential, long-term residential, and 
correctional facilities clients were the most likely to attend AA, NA, or CA meetings (on 
average), while methadone and non-methadone outpatient clients were the least likely. On 
balance, then, there appears to be an inverse relationship across modalities between treatment 
duration and treatment intensity. Within-modality variability on meeting attendance was 
substantial in all modalities, however, with SDU mean percentages ranging from 0 to 100 in non- 
methadone outpatient and long-term residential, and 3 to 100 in short-term residential 
(Appendix B, Exhibit B-3). Save for one short-term residential outlier, only methadone SDUs 
provided high percentages of clients with medications to treat their substance abuse problem 
(Appendix B, Exhibit B-4). 

The percent of clients who needed medical treatment and received it was highest in the 
two residential modalities (83 and 73 percent in short- term and long-term, respectively). The 
highest variability across SDUs within modality was in non-methadone outpatient programs, 
where means ranged from 0 to 100 percent (Appendix B, Exhibit B-5). Variability also was high 
in the methadone and long-term residential modalities. The percent of clients who needed 
mental health treatment and received it also was highest in the two residential modalities (32 and 
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Treatment Components by Modality 




METHADONE 


'NON-, 

METHADONE 

OUTPATIENT 


SHORT-TER -T 

RESIDENTIAL 


LONG-TERM 

RESIDENTIAL 


CORRECTIONAL 

FACILITIES 


Mean Length of Sta 
(days) 


302 


127 


50 


107 


70 


Mean Individual and 
Group Counseling 
(hours per month) 


5.1 


11.5 


17.0 


11 


17.6 


Attended AA, NA, or 
CA Counseling 


38% 


60% 


78% 


94% 


89% 


Received Substance 
and Alcohol 
Medications 


90% 


5% 


9% 


3% 


<1% 


Received TX for 
Medical Conditions 


44% 


46% 


83% 


73% 


67% 


Received TX for 
Mental Health 
Conditions 


17% 


23% 


32% 


31% 


25% 


Received HIV and 
TB Testing 


82% 


28% 


25% 


42% 


28% 


Mean % Received of 
Interpersonal, 
Family, Child Care, 
and Parenting Needs 


28% 


35% 


47% 


59% 


47% 


Mean % Received of 
Educational, Legal, 
Vocational, Transp. 
and Housing Needs 


22% 


34% 


27% 


58% 


38% 


SDU Matches All or 
Most Clients 


90% 


62% 


74% 


58% 


27% 


Client Saw His/Her 
Treatment Plan 


82% 


74% 


75% 


80% 


83% 


Client Given 
Analgesics 


2% 


2% 


19% 


12% 


8% 


Client Given Anti- 
anxiety Medications 


1% 


2% 


3% 


1% 


1% 
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31 percent in short-term and long-term, respectively). However, there was high variability across 
long-term residential SDUs on this dimension, with means ranging from 7 to 100 percent 
(Appendix B, Exhibit B-6). Variability also was also high across non-methadone outpatient 
SDU means (0 to 67 percent). Clients in methadone clinics were two to three times as likely to 
receive diagnostic HIV or TB testing than any other modality, while non-methadone outpatient 
and long-term residential facilities had the highest variability across SDUs (Appendix B, 

Exhibit B-7). 

On the family counseling, interpersonal, parenting, and child care services index as well 
as the high school, vocational, legal, financial, housing, and transportation services index, long- 
term residential clients received the most services (mean scores of 59 and 58 percent, 
respectively) while methadone clients received the least (28 and 22 percent). This is consistent 
with expectations from the literature that long-term residential care affords the best opportunit 
for providing ancillary or “wraparound” services once clients are in recovery, while outpatient 
methadone clinics (which primarily exist to provide methadone) tend to focus on these. In all 
modalities other than methadone, the variability across SDUs was higher on the first index than 
the second (See Appendix B, Exhibits B-8 and B-9). 

Methadone SDUs were the most likely to use patient matching as part of treatment 
(Exhibit III-l), while correctional facilities were the least likely to use matching. Mean 
differences between modalities on “patient saw treatment plan” were small (only 8 percentage 
points separated the highest from the lowest), although variability across SDUs within modalit 
differed substantially. Means in non-methadone outpatient SDUs ranged from 0 to 100 percent, 
compared to 64 to 91 percent in methadone SDUs (Appendix B, Exhibit B-10). 

2. CONTRIBUTION OF TREATMENT COMPONENTS TO LOGISTIC 
REGRESSION MODELS 

Over and above the client background variables, adding the treatment components 
significantly improved the models in non-methadone outpatient and short-term residential 
modalities for both outcomes and for methadone maintenance for substance treatment abstinence. 
Improvement was marginal for the long-term residential modality, with a p value of .054 for 
treatment substance abstinence and .095 for all substance abstinence. Including treatment 
components did not improve the models for correctional facilities. 

Exhibits III-2 and III-3 present odds ratios of the treatment components for analyses of 
treatment substance abstinence and all substance abstinence, respectively. The interactions 
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included for each modality appear under the variables from which they were constructed; for 
example, an interaction term calculated as the product of hours per month in counseling and 
percent of social/family counseling needs met appears twice in the exhibits under each treatment 
component. Exhibit III-4 presents estimates of the explained variance (pseudo R^) for 
background models and improvement due to the treatment components and interactions. 



EXHIBITin-2 ^^ ^ 

Treatment COMPONE^^^s AND Multivariate Odds Ratios OF 
Treatment Substance Abstinence 






;-:.:--:N0N-, . ^ , 












METHADONE 


SHORT TERM 


LONG-TERM 
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METHADONE 


OUTPATIENT 


RESIDENTIAL 


RESIDENTIAL 


FACILITIES 


Length of Stay 


L002 


1.004** 


1.000 


1.003* 


1.001 


With Treatment Completion 


- 


0.995* 


- 


- 


- 


Hours/mo in Group/Indiv 


0.901 


0.996 


1.044** 


0.933* 


1.009 


With %Met: Social/Family 


LOOl'^ 


- 


- 


1.001* 


- 


With %Met: Empl/Educ/Leg/Hsng 


- 


- 


0.999* 


1.001* 


- 


With Female Gender 


- 


- 


- 


0.924** 


- 


Attended AA, NA, or CA 


0.717 


1.071 


0.857 


1.671 


1.520 


With Treatment Completion 


- 


2.908** 


0.269* 


- 


- 


Received Substance and Ale Meds 


0.565 


0.554 


2.422* 


0.737 


0.759 


With Female Gender 


- 


- 


0.048** 


- 


- 


Medical Treatment 


1.333 


1.096 


0.743 


1.095 


0.884 


With Analgesics 


- 


- 


- 


3.481^^ 


- 


Mental Health Treatment 


0.645 


0.760 


0.860 


0.941 


0.845 


HIV and TB testing 


0.659 


0.844 


0.743 


0.572** 


1.140 


%Met: Social/Family 


1.001 


1.002 


1.001 


0.993 . 


1.002 


With Hours/mo in Group/Indiv 


l.OOl'^ 


- 


- 


1.001** 


- 


With Female Gender 


0.980* 


- 


- 


- 


- 


%Met: Empl/Educ/Leg/Hsng 


0.985* 


0.997 


1.018* 


0.995 


0.997 


With Hours/mo in Group/Indiv 


- 


- 


0.999* 


1.001* 


- 


SDU Matches All or Most Pts 


8.521* 


0.861 


1.929^^ 


0.469** 


3.980** 


Saw Treatment Plan 


3.052* 


1.097 


1 .674^^ 


1.126 


1.637^^ 


With Female Gender 


- 


- 


0.430^^ 


- 


- 


Received Analgesics 


2.032 


2.835 


0.751 


0.216* 


0.902 


With Medical Treatment 


- 


- 


- 


3.481^^ 




Received Antianxiety Meds 


0.639 


0.199* 


3.692* 


1.170 


8.537 


Avg. Methadone Dose 


1.003 


- 


- 


- 


- 



Interaction terms shown in italics. 
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Treatment Components AND Multivariate Odds Ratios of 
■ ■ ■ lAldSubstance!& ■ • 
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METHADONE 
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RESIDENTIAL 


FACILITIES 


Length of Stay 


1.001 
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- 
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0.793* 
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- 
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- 


- 


- 


0.955* 


- 
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Medical Treatment 


1.100 


1.057 


1.221 


0.813 
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With Analgesics 


- 


- 


- 


3.537 


- 


Mental Health Treatment 


0.507 


0.749 


0.780 


0.999 


0.928 


HIV and TB testing 


0.953 


0.732 


0.591* 


0.773 


1.151 


%Met: Social/Family 


0.999 


1.006** 


1.001 


1.007 


1.006* 


With Hours/mo in Group/Indiv 


1.003* 


- 


0.999* 


1.000 


- 
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0.990 


- 


- 


0.955 


- 


%Met: Empl/Educ/Leg/Hsng 


0.986 


1.000 


1.023** 


1.002 


0.997 


With Hours/mo in Group/Indiv 


- 


- 


- 


1.000 


- 


SDU Matches All or Most Pts 


19.94* 


0.900 


1.483 


0.507^ 


3.215* 


Saw Treatment Plan 


3.062 


1.211 


2.444* 


1.023 


1.323 


With Female Gender 


- 


- 


0.228** 


- 


- 


Received Analgesics 


1.927 


1.241 


0.834 


0.204* 


0.695 


With Medical Treatment 


- 


- 


- 


3.537 


- 


Received Antianxiety Meds 


- 


0.118^ 


4.479* 


0.553 


8.640* 


Ave. Methadone Dose 


0.999 




- 


- 


- 



Interaction terms shown in italics. 



*Model for Methadone Maintenance modality did not meet convergence criteria. 

p^O.lO. 

* p^0.05. 

**p^0.01. 
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METHADONE 
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Treatment 
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Abstinence 

Background 
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12,1 


Backgrounds Tre 
atment 


16,1 


14,7 


12,9 


11,6 


13,4 


All Substance 
Abstinence 

Background R~ 


NA 


5,3 


4,9 


8.6 


4,0 


BackgroundsTre 
atment R^ 


NA 


9.9 


11,3 


12,2 


5,6 



Below, we describe the effects separately by modality. 

2,1 Methadone 

For the methadone modality, SDU matching was significant and positive with high odds 
ratios for both outcomes (see Exhibits III-2 and III-3). Seeing the treatment plan was positive 
and significant at a 0.05 level for treatment substance abstinence and positive but not significant 
for all substance abstinence. Hours per month in group and individual counseling was negative 
for both outcomes but only significant for all substance abstinence (the p value for treatment 
substance abstinence was 0.0525). The percentage of needed family and interpersonal services 
received was not significant by itself, but the interaction between this component and counseling 
hours was significant and positive in all substance abstinence, tending to mitigate the negative 
effect of counseling when there is a high percentage of satisfied social and family counseling 
needs. 



The percentage of needed educational and other ancillary services received was 
significantly and negatively associated with treatment substance abstinence. This negative 
finding may suggest that these services compete with other priorities in a more medicall 



BEST COPY AVAILABLE 



■ '!SAT\NEDS\Components\TXCOMPON.WPD 



NEDS, Page 16 



Results 



intensive environment. As suggested in Exhibit III-3, such services may not be priorities in a 
methadone setting. Average methadone dose, applicable to this modality only, was not useful in 
predicting outcome. The average methadone dose for these clients, leaving out the effect of dose 
calibration in the first 2-3 weeks, is 45-50 mg. This is in the low range of what currently is 
regarded as effective (CSAT, 1993), and the variation in these data is somewhat limited. 

2.2 Non-Methadone Outpatient 

For the non-methadone outpatient modality, length of stay is positively and significant! 
associated with abstinence (both outcomes). This finding is mitigated by an interaction with 
treatment completion, however, which counters the effect of length of stay when clients complete 
treatment. Completion of treatment in a long-term outpatient setting may not mean the same 
thing as it does in other modalities. Treatment completion for this modality is positive (OR=1.7, 
p=.09 for treatment substance abstinence and OR=2. 1, p<.05 for all substance abstinence), and it 
is important to remember that the negative interaction effect of 0.994-0.995 only mitigates the 
other positive effects, suggesting that the combined effect is not simply additive. An interaction 
with treatment completion also was observed with AA, NA, or CA attendance during treatment, 
and although the main effect is not significant, it suggests that in this modality attendance at AA, 
NA, or CA meetings may be more beneficial among clients who complete treatment. The 
percentage of needed social and family services received is significant (p<.05) and positive for all 
substance abstinence but not for treatment substance abstinence, which implies that family and 
interpersonal supports may be most effective in achieving complete abstinence. Receiving anti- 
anxiety medications is negatively associated with both outcomes (at a significant level of .05 for 
treatment substance abstinence 0. 10 for all substance abstinence), possibly because of tolerance 
issues that emerge in long-term, outpatient settings or because clients who needed such agents 
were among the most difficult to treat. (Recall from Exhibit III-I that very few non-methadone 
outpatient clients reportedly received these medications.) 

2.3 Short-Term Residential 

Short-term residential modalities show positive, significant associations for both 
outcomes with hours per month in counseling and percentage of educational, vocational, and 
other ancillary services received, although an interaction between the two slightly reduces their 
combined effect. Short-term residential also is the only modality that presents positive and 
significant effects for substance and alcohol medications, although for all substance abstinence. 
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this is only significant at 0. 10. Receiving anti-anxiety medications also was positive and 
significant. Whether clients saw their treatment plans was significantly positive for both 
outcomes (increasing the odds of success by 67 and 144 percent on treatment and all substance 
abstinence, respectively). SDU matching was positive but significant only at 0. 10 and only for 
treatment substance abstinence, where matching increased the odds of success by 93 percent). 
One puzzling finding is the negative interaction between AA, NA, and CA attendance and 
treatment completion. This is in contrast to non-methadone outpatient SDUs where treatment 
completion in combination with AA, NA, and CA increases the odds for abstinence. Generally, 
those treatment components that are significant and positive for this modality imply better 
outcomes with more intensive treatment: more hours per month in group and individual 
counseling, more substance and alcohol medications, more ancillary services, and anti-anxiet 
medications. One complication to this picture is the interaction of gender with substance and 
alcohol medications and with treatment plans which suggests that men may have benefitted 
more from these medications and from seeing their treatment plaris. Note that the odds ratios in 
Exhibits III-2 andIII-3 are coded to describe women, and that the inverse describes the likelihood 
for men. Women account for 33 percent of clients in short-term residential facilities, and overall 
show a greater likelihood of abstinence (OR=3.3, p<.001 in both outcomes). 

2.4 Long-Term Residential 



Long-term residential modalities show positive and significant associations with length of 
stay for both outcomes. This is the only modality in which SDU matching is significant! 
negative (0.10 for all substance abstinent), but that may reflect differences in the way matching is 
accomplished and the purposes it serves in a long-term setting. This also is the only modalit 
with a significant and negative result for HIV and TB testing (treatment substance abstinence). 
Interactions were found for treatment substance abstinence between hours per month in group 
and individual counseling and the percentages of needed services (social/family and 
empl/educ/leg/hsng) that were received. For treatment substance abstinence, hours per month in 
abstinence and hours per month in counseling is significantly and negatively associated with 
abstinence, while high values both for hours in counseling and percentages of needed services 
that were received counter the effect. An interaction (p=.09) was observed between receiving 
analgesics and receiving medical treatment, suggesting that analgesics are associated with 
positive outcomes in combination with medical treatment but negative without medical 
treatment. 
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2.5 Correctional 

Correctional facilities modalities show positive and significant effects for SDU matching 
on both outcomes (increasing the odds of abstinence by a factor of three), and for the percentage 
of needed family and interpersonal services met in all substance abstinence. Anti-anxiet 
medications show a positive and significant effect for all substance abstinence. No significant 
interactions were uncovered for this modality. 
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IV. Summary and Conclusions 



This section of the report provides answers to the questions posed in the Introduction. 

Also presented are sections detailing the limitations that must be considered in interpreting the 

results and some implications for further analysis, policy, and practice. 

1. ANSWERS TO STUDY QUESTIONS 

1.1 What treatment components appear to improve outcomes, and hovt' do these 

relationships vary hy modality? 

■ Several treatment components were shown to have significant effects on the odds of a 
positive outcome both on treatment substance and all substance abstinence. Despite a 
difference of 439 clients, the effects of treatment components on treatment substance 
abstinence and on all substance abstinence are very similar. 

■ Length of stay shows a consistently positive effect both on treatment substance and all 
substance abstinence, consistent with expectations from the literature. However, the 
effects were statistically significant in the Non-methadone outpatient and Long-term 
residential modalities only. In these modalities, the odds ratios suggest that 
controlling for other variables, increasing length of stay by 30 days can improve the 
odds of a successful outcome by as much as 15 percent. The absence of length-of- 
stay effects in short-term residential settings is understandable, given that these all 
tend to be standard 28-day programs based on the chemical dependency model. 

■ Whether clients reported seeing their treatment plan was also consistently positive 
across all modalities, though statistically significant in only two. Seeing treatment 
plans significantly increased the odds of all-substance abstinence in Methadone 
modalities by 200 percent (OR=3.052) and by 144 percent (OR=2.444) in short-term 
residential modalities.^ 

■ Percent of needed interpersonal and family services received was mostly positive 
across outcomes and modalities but was significant only for all-substance abstinence. 

■ Asa main effect, hours per month in group and individual counseling had a 
significant positive effect in the short-term residential modality only. Educational, 
vocational and other ancillary services, the use of antianxiety and substance and 
alcohol medications, and client matching also had positive impacts on one or both 



An analysis of treatment retention using this data set showed seeing the treatment plan to be the most consistent 
predictor of program completion after controlling for other client and program factors (Orwin & Williams, 1999). 
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forms of abstinence in short-term residentials SDUs, suggesting that greater treatment 
intensity may pay off in that modality. 

■ Other significant findings varied greatly by direction and modality. Methadone SDUs 
showed positive and significant associations with SDU matching. Non-methadone 
outpatient SDUs showed positive effects of needed family, interpersonal, parenting 
and child care services that were met. Long-term residential SDUs showed negative 
effects for matching while correctional SDUs showed positive effects. As noted 
earlier, the negative relationship of matching in long-term residential SDUs ma 
reflect differences in the way matching is accomplished and the purposes it serves in a 
long-term setting. It also may mean that matching is more prevalent in SDUs serving 
more difficult clients. 

■ One difference between the two endpoints is the significance and strength of social 
and family services in non-methadone outpatient and correctional facilities 
modalities. This component is significant and positive in all-substance abstinence but 
not in treatment-substance abstinence. Much of this is due to poorer model fit for 
short-term residential and correctional facilities modalities for all-substance 
abstinence, as shown in Exhibit III-4. However, it may also suggest an investigation 
as to whether interpersonal skills or family supports are more effective for total 
abstinence and more useful in reducing cross-addiction and substance substitution. 

1.2 What interactions occur between client-level factors and treatment 
components? 



■ Significant interactions were observed between hours in group or individual 

counseling with the percent of needed social/family services received and the percent 
of needed employment/educational services received. Generally, the interactions with 
these two indices tend to mitigate the effects of hours per month in group or 
individual counseling, in that the interaction terms are positive when the effect of 
hours per month is negative (methadone, long-term residential) and negative when 
the effect of hours per month is positive (short-term residential). An additional 
interaction (significant at p<.10) was found between receiving analgesics and 
receiving medical care for the long-term residential modality. 



■ Background variables found to interact with treatment components were gender and 
treatment completion. While women were significantly more likely to be abstinent in 
most modalities, the positive effects of hours per month in counseling, substance and 
alcohol medications, percent of needed services received, and treatment plans 
appeared to differentially benefit men. This suggests that providers may need to offer 
more tailored approaches to these services so that women benefit as well. Studies 
indicate that women clients are more responsive to interventions aimed at self-esteem. 
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anxiety, and depression (Bartholomew, Rowan-Szal, Chattam, & Simpson 1994), so 
that adjusting for women’s overall rate of abstinence may have left a residual group 
resistant to more intensive treatment or other kinds of treatments. 

■ Treatment completion interacts with length of stay (non-methadone outpatient) and 
AA/JNA/CA attendance (non-methadone outpatient and short-term residential). In the 
case of meeting attendance in the non-methadone outpatient modality, the main effect 
was not significant while the interaction was positive, suggesting that in this modalit 
attendance at meetings may be more beneficial among clients who complete 
treatment. This may reflect the fact that meeting attendance and treatment completion 
are both manifestations of a single unmeasured factor such as motivation, which in 
turn may be most likely to affect outcomes in the relatively unstructured environment 
of outpatient treatment. However, the same interaction was negative in the short-term 
residential modality, for reasons that are not readily apparent. As noted in Chapter II, 
Methods, no specific interactions were predicted on the basis of the literature, and 
caution should be exercised in interpreting them. 



1.3 How much do the treatment components improve the prediction of outcomes? 

Compared to the background variables alone, the inclusion of treatment components 
significantly improved the logistic regression models for non-methadone outpatient and short- 
term residential modalities for both outcomes and for methadone for treatment substance 
abstinence. Improvement for the long-term residential modality was marginal (p=.054 for 
treatment substance abstinent and p=.095 for all substance abstinent) and not significant for the 
correctional modality. Using the background models as the base, adding the treatment 
components increased the estimated variance accounted for by 1-2 percent in correctional 
modalities and 3-8 percent in the other modalities. 



2. LIMITATIONS 



The following limitations should be considered in interpreting these results. Several 
sensitivity analyses were conducted to assess the potential extent of the limitations on the 
findings, which are also described: 
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■ In some instances, limitations in the detail in NTIES questionnaires limited the specificit 
with which treatment components could be defined. For example, reports by clients in the 
exit questionnaire did not distinguish group and individual counseling, so the two were 
combined. Also, incomplete responses affected how a component was defined and 
developed. Several variables were included in modeling because of their treatment 
importance, but were defined as simple yes/no dichotomies because no further details were 
available. For example, clients reported going to Cocaine Anonymous (CA), Narcotics 
Anonymous (NA), or Alcoholics Anonymous (AA) during treatment without distinguishing 
between the three or reporting the number or frequency of meetings. Similarly, the 
reported taking prescribed substance and alcohol medications during treatment but did not 
identify them. Matching appears in substance abuse and treatment literature as a means of 
determining the least restrictive treatment environment, as a way to ensure that needed 
services are provided once a client has been placed in treatment, and as a way to select 
counselors or primary providers that can best meet clients’ special needs. The NTIFS data 
set has no information concerning the manner in which clients found their way into their 
respective SDUs. The data set provides only limited information on the reason for 
matching and other details once clients were admitted, and does not directly specify which 
clients were matched. The use of treatment plans also appears both in the literature and in 
the NTIFS data; however, the manner in which the item was interpreted, particularly in the 
Baseline Administrative Report (NEAR), is unclear, requiring this analysis to focus on 
whether the client recalled seeing a treatment plan. 



■ Inferences from non-experimental data are inherently vulnerable to selection bias. In 
such situations, it is customary to model background variables — e.g., demographics — as 
a proxy for the selection process (case-mix adjustment). However, this approach has 
technical drawbacks. Ideally, the problem is addressed by constructing selection 
models and measurement models and analyzing the data with a structural equation 
modeling program such as LISRFL or FQS. However, the requirements of these 
models were not met in these data. As an alternative, a hierarchical “set” approach was 
employed, with the background variables set entered prior to the treatment variables set. 
The hierarchical approach produces a conservative estimate of the treatment effects, 
because outcome variance common to both the background variables and the treatment 
variables are “credited” to the background variables. That is, the background variables 



First, it is vulnerable to model misspecification. That is, unless the true selection process is completel 
“explained” by variation in the background variables (predictors), the adjusted effect estimates still may be biased. 
Second, the improper adjustment is further exacerbated by measurement errors (unreliability) in the background 
variables. Such errors bias the effect estimate because the expected value of the adjusted mean difference under 
the null hypothesis (i.e., no effect) is nonzero. In practice, the analyst typically will not know whether the, 
background variables in a particular analysis over adjust, under adjust, or properly adjust. Apart from internal 
consistency tests on the severity scales (Cronbach’s alpha), the NORC report reported no formal reliabilit 
assessments of patient self-reports at intake. 
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are given causal priority over the treatment variables in the hierarchy. The hierarchical 
approach therefore provides a measure of protection against the misinterpretation of 
selection bias as treatment effects. 



■ An analogous issue exists with treatment components. The treatment components at 
many SDUs look nominally similar: e.g., individual therapy, group therapy, and 
assorted wraparound services. Yet the way the services are configured 
(e.g., bundling, sequencing, etc.) may vary, and the variation may reflect attempts to 
attain different treatment goals and use different therapeutic emphases. However, this 
variation will not be apparent in the services data themselves. Consequently, the 
variation that exists between SDUs may be under-specified in the regression models, 
which can in turn lead to biased estimates of effect parameters." As a personal check 
against this, we included an additional set of variables representing goals and 
emphases in the models." These variables were added to the models as a third set in 
the sequence, after including the client factors and treatment components. As such, 
this functioned as a sensitivity analysis with the variables serving as proxies for 
unmeasured residual treatment effects that were not being captured by the observed 
treatment components. None of the variables significantly contributed to model fit, 
and in fact many could not be modeled because they were linear combinations of 
other components. To the extent that this additional set of variables were valid 
proxies for unmeasured residual treatment, they offer no evidence that unmeasured 
treatment is biasing effect parameters by causing under-specification of models. 
However, the possibility remains of unmeasured residual treatment not captured b 
these variables, the extent and effects of which remain unknown. 



■ A related issue is whether to measure total treatment only as it was provided and 
reported by the SD, or to describe dosage received by the SDU and other agencies to 
which clients were referred. Questions in the NTIES Treatment Experience 
Questionnaire (NTEQ) ask whether services were provided at the SDU, at a facilit 
referred to by the SDU, or at an unknown facility. Since this analysis focuses on the 
effect of treatment components on outcomes without considering the impact of 
resources or sponsorship, a client was deemed to have received a service even if it 
was received “somewhere else.” The use of client-reported information thus provides 
additional information on treatment that would not be reported in SDU-level data. 



" Measurement error also may be present from differences in the way program administrators interpreted treatment- 
related questions on the NEAR (e.g., patient matching). The NORC report also cautioned about the accuracy of 
any single source of information regarding services delivered or received (NORC, 1997). 

'• The baseline administrative interview asked program directors a series of questions regarding 12 potential 
treatment goals and 12 types of counseling and therapy. It also asked program directors to what extent their 
programs emphasized each potential goal of treatment and to rank the 5 most important treatment goals. 

Similarly, they were asked what degree of emphasis programs placed on each type of counseling or therapy. 

We converted each treatment goal into a dichotomous variable indicating whether it was among the top 5. 
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However, because of the sketchy nature of these “somewhere else” responses, a 
comparative sensitivity analysis was done that limited most services to those done at 
the SDU or referred by the SDU. Re-running these analyses after removing services 
received “somewhere else” made little impact upon results. 



■ Problems in convergence for all substance abstinence for methadone and the limited 
improvement in model fit for the correctional modality limit the confidence that can 
be placed in findings for these modalities. 



■ Perhaps the most significant limitation is the inability to place the SDU episode in the 
context of the client’s entire treatment history. Any attempt to compare treatment 
received with treatment intended ideally should account for what providers consider 
their mission, and which service providers feel to be part of that mission. Clients 
referred into or out of one of the participating SDUs because a phase of treatment is 
not provided may present an incomplete history. Twenty-one percent of clients with 
follow-up received services from another SDU within 7 days of intake and most 
received detoxification services. Treatment provided by an SDU may be 
misunderstood as incomplete or inadequate when actually it was done somewhere else 
and not reported. The administrative questionnaire asks SDUs which phases of 
treatment the SDU provides routinely from a list of (1) detoxification, 

(2) rehabilitation, (3) assessment/intake, (4) steps (from 12 Steps), (5) aftercare, 

(6) relapse prevention, and five “other specified” phases. Most SDUs said they divide 
treatment into stages or phases, and that phases they do not provide may be done at 
another, “linked” service provider. Two-thirds of the clients used in this analysis 
were treated at SDUs that were “considered one phase of treatment in a sequence of 
closely affiliated SDUs.” Clients may have received additional treatment that is not 
included in the NTIES data set. 

3. IMPLICATIONS FOR FURTHER ANALYSIS, POLICY, AND PRACTICE 

3.1 Analysis 

The following lists areas for further research that could productively be investigated with 
the NTIES data: 

■ This analysis focused entirely on treatment components reported in the administrative 
report, the intake questionnaire, the treatment exit questionnaire, and the patient 
abstract. Follow-up instruments were polled for outcome data only. Numerous 
articles in the literature describe aftercare as a crucial component (e.g., Cougheym, 
Feighan, Cheney, & Klein, 1998; Hubbard, et al., 1989; Lash and Blosser, 1999) and, 
in fact, aftercare is cited as a phase of treatment by many providers. Examination of 
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this treatment component will require use of the follow-up to investigate and quantif 
post-treatment services such as AA, NA, and CA meetings. 



■ Studies also emphasize the importance of urinalysis testing during treatment (CSAT, 
1995). This was omitted because it is not specifically a treatment component; 
however, the literature presents this issue as a means of instilling accountability and 
managing client noncompliance. This issue warrants further study. 



■ Assessment at intake is also prominent in literature, not as a specific component of 
therapy but as an overall phase of treatment that is closely related to the use of 
treatment plans and patient matching. This aspect of treatment should be examined in 
detail, with emphasis on the interplay between assessment activities at intake and the 
use of treatment plans and client matching. 



■ Overutilization of services is a salient issue in today’s fiscal environment, with more 
private and public sector treatment funding shifting to managed care models. In 
constructing indexes for the present analysis, clients who received specific services 
were assumed to need them even if they did not express a need for them at intake. 
This was reasonable given that clients are not the best assessors of their own service 
needs, particularly at program intake when many are not yet into recovery. However, 
an alternate scenario — clients receiving services they did not need and would not 
benefit from — is also a possibility, and its extent and impact could be investigated in 
these data. 

■ As stated earlier, the intent of the present analysis was to define treatment 
components in ways that have external applicability beyond the NTIES data, using 
variables to define the components that treatment providers could readily manipulate. 
This does not preclude the existence of other and possibly better ways that treatment 
can be conceptualized and operationalized in these data. It would be of particular 
interest to see whether alternate conceptualizations yielded similar results; such a 
finding would increase confidence in the results reported here. 

3.2 Policy and Practice 

These results pose the following implications for agencies involved in funding and 
delivering substance abuse treatment: 



■ The positive relationships with hours per month in group and individual counseling, 
the percentage of needed educational and vocational counseling received, and 
substance and alcohol medications, suggest that greater treatment intensity in short- 
term residential SDUs can improve treatment outcomes. 
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■ Lx)nger lengths of stay may enhance treatment outcomes in non-methadone outpatient 
and long-term residential modalities, even when patients do not complete treatment. 



■ Client participation in the development of their treatment plans can potentiall 

enhance treatment outcomes with little cost or risk. We found little evidence that it 
was harmful or that it diverted resources from other treatment strategies, and evidence 
that it was generally helpful in all modalities. 



■ Matching clients to counselors or key providers appears to improve treatment 

outcomes in methadone maintenance and correctional modalities but not in long-term 
residential facilities. 



■ The apparent effectiveness of treatment components like counseling hours per month, 
employment and vocational services, and substance and alcohol medications var 
considerably by modality. This suggests that modality should be taken into account 
before recommending implementation or expansion of these components. 
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Description of the National Treatment Improvement 
Evaluation Study and Center for Substance Abuse 
Treatment Demonstrations (1990-1992) 

The National Treatment Improvement Evaluation Study (NTIES) was a national 
evaluation of the effectiveness of substance abuse treatment services delivered in comprehensive 
treatment demonstration programs supported by the Center for Substance Abuse Treatment 
(CSAT). The NTIES project collected longitudinal data between FY 1992 and FY 1995 on a 
purposive sample of clients in treatment programs receiving demonstration grant funding from 
CSAT. Client-level data were obtained at treatment intake, at treatment exit, and 12 months after 
treatment exit. Service delivery unit (SDU) administrative and clinician (SDU staff) data were 
obtained at two time points, 1 year apart. 

1. THE NTIES DESIGN 



The NTIES study design had two levels — an administrative or services component and a 
clinical treatment outcomes component. 

1.1 The Administrative/Services Component 

This study component was designed to assess how CSAT demonstration funds were used, 
what improvements in services were implemented at the program level, and what kind and how 
many programs and clients were affected by the demonstration awards. 

The unit of analysis for the administrative component was the SDU, defined by CSAT as 
a single site offering a single level of care. The classification of level of care is based on three 
parameters; (1) facility type (e.g., hospital, etc.); (2) intensity of care (e.g., 24-hour, etc.); and 
(3) type of service (e.g., outpatient, etc.). An SDU could be a stand-alone treatment provider or it 
could be one component of a multi-tiered treatment organization. For example, a large count 
mental health agency may be the organization within which the SDU is located. The 
organization may have multiple substance abuse treatment components, such as a county hospital 
and a county (ambulatory) mental health center. The county hospital may have multiple SDUs, 
such as an inpatient detoxification service, an outpatient counseling service, and a hospital 
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satellite center providing transitional care. In summary, the SDU provided NTIES evaluators 
with a stable, uniform level of comparison for examining service delivery issues. 

1.2 Clinical Treatment Outcomes Component 

The unit of analysis for the clinical treatment outcomes component was individual client 
data. NTIES measured the clinical outcomes of treatment primarily through a “before/after” or 
“pre- to post-treatment” design. This method compares behaviors or other individual 
characteristics in the same participants, measured in similar ways, before and after an 
intervention. 



Information about clients’ lives for the before period were obtained from the NTIES 
Research Intake Questionnaire (NRIQ), which was administered sometime during the clients’ 
first 3 weeks of treatment. The specific areas assessed included: 



■ Drug and alcohol use 

■ Employment 

■ Criminal justice involvement and criminal behaviors 

■ Living arrangements 

■ Mental and physical health. 

Information about clients’ lives for the after period were obtained from the NTIES Post- 
discharge Assessment Questionnaire (NPAQ), with the same areas assessed at roughly 12 months 
post-treatment. Other client data sources included a treatment discharge interview (NTIES 
Treatment Experience Questionnaire, NTEQ), abstracted client records, urine drug screens 
collected at the time of the follow-up interview, and arrest reports from state databases. 
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1.3 The Outcome Analysis Sample 

Between August 1993 and October 1994, research staff successfully enrolled 6,593 
clients at 71 SDUs to participate in three waves of an in-person, computer-assisted data 
collection protocol. These SDUs were chosen from the universe of treatment units receiving 
demonstration grant funding from CSAT. Some of the selected facilities were wholly supported 
by CSAT awards, while others received only indirect support or none. 



Clients were interviewed at admission to treatment, when they left treatment, and then at 
12 months after the end of treatment. Less than 10 percent of the recruited clients refused or 
avoided participation, and more than 83 percent of the recruited individuals (5,388 clients) 
completed a follow-up interview. Additional sample exclusions included: 



■ Missing or undetermined treatment exit date 

■ Inappropriate length of follow-up interval (less than 5 or more than 16 months) 

■ Clients incarcerated for most or all of the follow-up period. 

The additional sample exclusions resulted in a final outcome analysis sample of 4,41 1 
individuals. 

2. TREATMENT DEMONSTRATION PROGRAMS 



CSAT initiated three major demonstration programs and made 157 multi-year treatment 
enhancement awards across 47 states and several territories during 1990 through 1992. One 
objective common to all demonstrations was CSAT’s emphasis on the provision of 
“comprehensive treatment” services to targeted client populations. The recipients of these 
awards focused special attention on the substance abuse treatment service needs of minority and 
special populations located primarily within large metropolitan areas. The demonstration 
programs are briefly described below. 
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2.1 Target Cities 



Under this demonstration, nine metropolitan areas were selected to receive awards, of 
which half were included in the NTIES purposive sample. The following treatment improvement 
activities were explicitly provided for in the awards: 



■ Establishment of a Central Intake Unit (CIU) with automated client tracking and 
referral systems in place 

■ Provision of comprehensive services, including vocational, educational, biological, 
psychological, informational, and lifestyle components 

■ Improved inter-agency coordination (e.g., mental health, criminal justice, and human 
service agencies) 



■ Services for special populations — adolescents, pregnant and postpartum women, 
racial and ethnic minorities, and public housing residents. 



2.2 Critical Populations 



o 



Under this demonstration program, awardees were required to implement “model 
enhancements” to existing treatment services for one or more of the following critical 
populations: racial and ethnic minorities, residents of public housing, and/or adolescents. 

Special emphasis was given to services provided to the homeless, the dually diagnosed, or 
persons living in rural areas. A total of 130 grants were awarded, covering services such as 
vocational support/counseling, housing assistance, integrated mental health and/or medical 
services, coordinated social services, culturally directed services, and others. 

2.3 Incarcerated and Non-Incarcerated Criminal Justice Populations 

Under this demonstration program, funds were directed toward improving the standard of 
comprehensive treatment services for criminally involved clients in correctional and other 
settings. Some program emphasis was placed on ethnic and/or racial minorities. Nine 
Correctional Setting demonstrations were funded: five in prisons, three in local jails, and one 
across a network of juvenile detention facilities. All projects included a screening component to 
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identify substance-abusing inmates, a variety of targeted treatment interventions (e.g., therapeutic 
communities, intensive day treatment programs), and a substantial aftercare component. 



A total of 10 non-incarcerated projects were funded. Five programs targeted 
interventions at clients in diversionary programs, three focused services on probationers or 
parolees, and two programs targeted both populations. Almost all of the funded demonstration 
projects included the following components: 

■ Basic eligibility determination, followed by systematic screening and assessment 

■ Referral to treatment 

■ Graduated sanctions and incentives while in treatment 

■ Intensive supervision in treatment 

■ Community-based aftercare with supervision and service coordination. 

In total, 19 criminal justice projects were funded as part of the CSAT 1990-1992 demonstrations, 
and as indicated in the next section, these projects were purposively over-sampled in order to 
obtain a more robust evaluation of this program.. 

3. DESCRIPTION OF SDUs AND CLIENTS BY TREATMENT MODALITY AND 
PROGRAM TYPE 

The 71 SDUs contributing clients to the outcome analysis sample are characterized b 
modality and (demonstration) program type in Exhibit A-1 below. Among the 698 SDUs in the 
NTIES universe: 52 percent (n=365) were Target Cities programs, 39 percent (n=274) were 
Critical Populations programs, and 9 percent (n=59) were Criminal Justice programs . 



In terms of the SDUs sampled for the NTIES outcome analysis, 44 percent were Target 
Cities programs, 38 percent were Critical Populations programs, and 23 percent were Criminal 
Justice programs. Criminal Justice SDUs were purposely over-sampled as part of the NTIES 
evaluation design (CSAT, 1997). Nearly half of the sampled SDUs were (non-methadone) 
outpatient programs, and about one-quarter were long-term residential programs. 
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;V Exhibit^ 

SDUs IN THE Outcome Analysis Sample 


^^ogram Title 
Number of SDUs 
(% of NTIES 
Universe)^^ 


NTIES 

Sample 


Methadone 


Outpatient 


Long-Term 

Residential 


Short-Ter 

Residenti^ 


Correctional 


Target Cities 

n=365 (52%) 


31 

(44%) 


6 


15 


6 


4 


0 


Critical Populations 

n=274 (39%) 


27 

(38%) 


1 


13 


10 


3 


0 


Criminal Justice 

n=59 (9%) 


13 

(23%) 


0 


5 


0 


0 


8 


Totals 

N=698(100%) 


71 

(100%) 


7 


33 


16 


7 


8 



As shown in Exhibit A-2, 59 percent of all NTIES clients were sampled from Target Cities 
SDUs. Slightly over 21 percent of all NTIES clients were sampled from Critical Populations 
SDUs and 20 percent were sampled from Criminal Justice SDUs. Outpatient (non-methadone) 
SDUs treated over one-third (35%) of the clients in the outcomes analysis sample, and almost 80 
percent of these were sampled from Target Cities programs. 



Exhibit A-2 

Distribution OF Clients IN THE Outcomes Analysis Sample 


Program Title 






Long-Term 


Short-Ter . 




Number of Clients (% of Analysis Sample) 


Methadone 


Outpatient 


Residential 


Residential 


Correctionail 


Target Cities 


377 


1,214 


504 


505 


0 


n=2,600 (59%) 


(89%) 


(78%) 


(60%) 


(58%) 




Critical Populations 


45 


220 


298 


368 


0 


n=931 (21%) 


(11%) 


(14%) 


(35%) 


(42%) 




Criminal Justice 


0 


132 


39 


0 


709 


n=880 (20%) 




(8%) 


(5%) 




(100%) 


Totals 












n=4,411 (100%) 


422 


1,566 


841 


873 


709 



BEST COPY AVAILABLE 

' The original NTIES universe of SDUs included a program type called Specialized Services. Because clients for 
the outcome analysis sample were not drawn from these SDUs (n=94), they are excluded from the Exhibit. 
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Distributions of SDU Means for 
Treatment Components Within Modalities 
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EXHIBIT B-2 

HOURS/MONTHS WITH PRINCIPAL PROVIDER: 
DISTRIBUTION OF SDU MEANS BY MODALITY 
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Methadone Non-Methadone Short-Term Long-Term Correctional 

Outpatient Outpatient Residential Residential 

Modality 



EXHIBIT B-3 

PERCENT Clients Attending aa, na, or CA meetings: 

DISTRIBUTION OF SDU MEANS BY MODALITY 
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PERCENT Clients receiving medications to Treatment Substance abuse problem: 

DISTRIBUTION OF SDU MEANS BY MODALITY 
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Methadone Non-Methadone Short-Term Long-Term Correctional 

Outpatient Outpatient Residential Residential 

Modality 

SDU Mean Modality Mean 



EXHIBIT B-5 

PERCENT Clients needing medical Treatment who received it: 
Distribution of SDU Means by modality 
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EXHIBIT B-6 

PERCENT Clients needing mental Health Treatment Who received it: 
DISTRIBUTION OF SDU MEANS BY MODALITY 
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Methadone Non-Methadone Short-Term Long-Term Correctional 
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Modality 
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EXHIBIT B-7 

PERCENT CLIENTS RECEIVING DIAGNOSTIC HIV OR TB TESTING: 
DISTRIBUTION OF SDU MEANS BY MODALITY 







O 

r- 



Q' 'SATVNEDSVComponentsMXCOMPON.WPD 




NEDS, Page B-7 



Methadone Non-Methadone Short-Temi Long-Term Correctional 

Outpatient Outpatient Residential Residential 

Modality 

SDU Mean Modality Mean 



EXHIBIT B-8 

FAMILY Counseling, interpersonal, parenting, and Child Care Services index: 

DISTRIBUTION OF SDU MEANS BY MODALITY 
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EXHIBIT B-9 

HIGH SCHOOL, VOCATIONAL, LEGAL, FINANCIAL, HOUSING & TRANSPORTATION 

distribution of sdu Means by modality 
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EXHIBIT B-10 

Percent clients reporting Seeing Individual Treatment plans: 

DISTRIBUTION OF SDU MEANS BY MODALITY 
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Technical Notes 



Appendix C 

Technical Notes 



This appendix contains further detail on determining the analyzable sample and 
operationalizing treatment components. It is primarily intended for readers wishing to replicate 
these analyses or conduct further analyses on the NTIES data using the same variables. 

1. SAMPLE SIZE 



As noted previously, a total of 3,085 clients were used in the analysis, after excluding 451 
clients from SDUs with no baseline administrative (NEAR) information, 730 clients without an 
exit questionnaire (NTEQ), and 145 clients with unknown information for any of the variables 
included in final models. Generally, variables with 30 percent missing data were excluded from 
analysis, but this is not always possible when several variables describing relatively rare events 
and presenting ambiguous missing codes are combined for analysis. This analysis was 
constructed using roughly 550 variables drawn from the total of 2,900 variables in the NEAR, 
NRIQ (intake questionnaire), NPRF (patient record abstract form), NTEQ and NPAQ (follow-up 
questionnaire). However, many of those variables comprise checklists that must be used as a 
block, leaving about 300 separate data elements of interest in the analysis. There were only one 
or two items with more than 30 percent missing for the NRIQ, NPRF, and the NPAQ, which 
contained data from interviews with the client. For the NPRF, which was usually filled out b 
SDU personnel, about 5 percent of variables had to be dropped. For example, the primar 
diagnosis at NTIES admission was unknown or missing for 60 percent of all clients. Moreover, 
additional missing data are embedded within blocks of variables that are presented together. For 
example, item FI 5 presents “services received” as a list of 24 sub-items. It asks the number of 
times each was provided, but does not provide an explicit “no” or zero response and has “DON’T 
KNOW/NO RECORD” for large portions of cases. Eecause the services are presented in a 
block, non-responses may be interpreted as “no” (instead of unknown) if there is a response to at 
least one of the 24 services. However, this interpretation leads to the conclusion that 78 percent 
of clients received no psychiatrist or psychologist visits (0), 77 percent received no employment 
counseling (#9), and 62 percent received no interpersonal skills training (#14). These data were 
found to be at odds with those cited in the client-reported treatment exit questionnaire (NTEQ), 
and therefore were used only when NTEQ data were not available. 
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2. IDENTIFYING CLIENTS WITH MENTAL HEALTH NEEDS 



Identifying dually diagnosed clients proved difficult. NTEES instruments record first and 
second DSM III diagnoses, but usually contained a second drug preference rather than a discrete 
co-morbidity. NRIQ questions about mental health symptoms focused on anxiety, depression, 
suicide or suicide ideation, hallucinations, and attention-deficit disorder (ADHD) and asked 
whether these symptoms occurred within the last 12 months and whether they were caused b 
drugs or alcohol. These general categories of mental illness were considered present if the 
condition or symptom was reported for the previous 12 month period and if the respective 
question about drug and/or alcohol involvement was answered “none”(hallucinations and ADHD 
were not restricted to the previous 12 months). A variable describing mental illnesses at index 
that were not reportedly caused by drugs or alcohol was defined to be “yes” if any of these 
general categories were present. This variable identified 813 clients from among the 4,41 1 with 
follow-up, ranging from 10 percent for methadone SDUs, 22 percent for non-methadone 
maintenance SDUs, and 15-19 percent for the remaining SDUs. Clients identified as having 
mental health disorders using this measure exhibited an average value of 34.6 on the mental 
health severity scale (RPSYSEV) compared to 17.4 for clients identified as having no mental 
health disorders. 

3. IDENTIFYING CLIENTS UNDER LEGAL PRESSURE TO SEEK TREATMENT 



In addition to the 709 clients in correctional facilities modalities, 1,007 clients in the 
other modalities reported being on parole or probation (F8 - NPRF). Nearly half of the clients in 
the file (1,924) expressly said they were not on parole or probation, and these data were missing 
for the remaining clients. Several relevant NRIQ items were reviewed that bear on possible legal 
pressure that might affect a client’s motivation and commitment to treatment, but ultimately the 
client’s self reported information was taken from NRIQ Item 373, which asks, “Is your coming to 
[AOD treatment] required or recommended by an attorney or anyone in the criminal justice 
system?” 

4. QUANTIFYING INDIVIDUAL AND GROUP COUNSELING (hours per month) 



Total time spent in counseling sessions is addressed in Section G of the Baseline 
Administrative Report (NEAR), Item 15 of the Patient Record Abstract (NPRF), and in sections 
of the Treatment Exit Questionnaire (NTEQ). The selection of variables to most accuratel 
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describe a client’s total exposure to counseling was not straightforward, however, due to missing 
and incomplete values in these data. 

Item Go of the NEAR asks for the number and duration of group counseling sessions. It 
was discovered that the SDUs for 1,764 clients provided detail for individual counseling but did 
not respond to the questions about group therapy. There are even fewer data for time in the 
classroom, with only 775 clients from SDUs that provided the information. 



Combining group and individual counseling was explored as an optional measure b 
assuming the disparity in response to these items resulted from confusion in their definitions or 
an inability to distinguish the two. However, this composite did not compare well with measures 
of total counseling time on other instruments. 

The NPRF matrix of 24 services in Item 15, “Services Received,” includes individual, 
group and family counseling among the listed services. These data were reviewed despite the 
large numbers of apparent non-responses to services like employment counseling and visits with 
psychiatrists. Of the 3,681 clients with follow-up and an NPRF form, 13 percent reported no 
group counseling and 8% reported no individual counseling. It seems doubtful that these 
13 percent and 8 percent actually mean that the SDU’s do not provide group or individual 
counseling, especially considering that comparable data from clients’ NTEQs indicate the 
received counseling. Furthermore, these items report the total number of times the services were 
received during treatment, rather than the number of hours per week clients took part in the 
activity. 



Item 176 of the NTEQ asks the client which “staff member [was] the most important to 
you” from those staff members the patient usually saw for counseling or treatment. Ninety-three 
percent of clients with follow-up indicated there was such a person and provided the number and 
duration of sessions with the provider, while 7 percent said that they could not single out one 
individual. This information was used in NORC’s analysis of client satisfaction. 



A measure of the time spent in group and individual counseling, calculated in hours per 
week, was constructed from all three data sources. Information about the primary provider from 
the NTEQ was considered first because it was more complete and allowed comparison with the 
client satisfaction analysis. However, use of this data required eliminating distinctions between 
group and individual counseling. Items 183 and 191 ask about the frequency (5+/wk, 2-4/wk, 
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1/wk, 2-4/mo, 1/mo) and duration (<10min, 10-29min, 30-59min, l-2hrs, more than 2hrs) of 
counseling or other treatment received from the clients’ principal providers. A quantitative 
variable calculated as the product of interval midpoints was calculated, deriving a mean of 12.6 
patient hours per month (median=9 pt hrs/mo), based on 3,402 of the total 4,41 1 clients with 
follow-up (730 clients have no NTEQ forms and 279 could not provide a principal provider). 
When this NTEQ data was unavailable, information was taken next from the NPRF matrix of 
services. Allowable responses include 1 time, 2-3 times, 4- 10 times, 1 1 or more times, “number 
uncertain” and “not mentioned in record.” These data were quantified by using the mid-point of 
the interval. The mean of the non-zero responses for that modality was computed and used for 
those clients who reported “number uncertain.” Values were converted into hours per month b 
using clients’ LOS and assuming sessions to be one hour in duration. Roughly 100 clients with 
neither a primary provider nor NTEQ data were assigned session hours per month based on the 
combined group and individual counseling reported in the NEAR. This measure was calculated 
as the product of the number of sessions and their duration as was done for the NTEQ data. 

5. IDENTIFYING CLIENTS FROM SDUs THAT EMPLOY MATCHING 



As noted previously, the NTIES data includes no information how patients came to be 
assigned to their respective modalities nor how they came to be enrolled in their specific SDU 
within the modality. Matching in this data set only addresses the SDU’s practices matching 
patients to treatment and counselors within their program. Within the NTIES data set, client 
treatment matching is addressed only in two items in Section F, Treatment Planning, of the 
Administrative Baseline Report (NIBAR). 



All of the SDUs that reported they matched patients also answered question F I Ob 
(B253M1-B253M8), which asks whether race, ethnicity, language, gender, age, drug use history, 
mental health needs or other criteria were used to match clients (categories are not exclusive). 
Among the 2,843 clients from SDUs reporting some kind of matching practice, 43 percent are 
from SDUs that match on the basis of race, 40 percent on the basis of ethnicity, 55 percent on the 
basis of language, 66 percent on the basis of gender, 22 percent on the basis of age, 41 percent on 
the basis of drug-use history, 46 percent on the basis of mental health, and 27 percent on the 
basis of other criteria. Battelle reviewed combinations of these eight ways of matching patients 
to treatment, and found that a third of clients were from SDUs that matched on five or more 
criteria. Matching as practiced by these SDUs may be a more developed and formal practice than 
that by SDUs that match on 1 or 2 obvious features. A total of 477 clients are from non- 



Q :SAT\NEDS\Componenls\TXCOM PON . WPD 



NEDS, Page C-4 



Appendix C 



methadone outpatient and long-term residential modalities that match clients on the basis of 
gender only, and 105 are from short-term residential modalities that only match on the basis of 
language. Another 227 are from SDUs that specified “other” criteria exclusively. It is not clear 
that these circumstances are comparable to the detailed assessment of needs talked about in the 
literature. Since there was little guidance in TIPS or other materials pointing to one matching 
criteria over another, for the purpose of the present analysis, clients from SDUs that matched on 
any of the F I Ob criteria were considered to have been matched if the SDU reported it matched 
most or all of its patients. 

6. CALCULATION OF AVERAGE WEEKLY METHADONE 



Average weekly methadone dosage was used as a treatment component for the methadone 
modality only. Methadone dosage is recorded in Item 17 of the NPRF with date and daily dose in 
milligrams. As noted previously, the manner in which the information was filled out depended 
upon the length of time the client was in treatment, so that a client’s dose might be recorded 
every day, week or month. For each client, these dose histories were converted into a 2-year or 
105 week array to provide a standard context regardless of how it was reported. It was then 
assumed that dosage within these “benchmarks” could be estimated from the known dosages. 

This was done by extending the reported dose for a week to the subsequent weeks, until the next 
reported dose was encountered. 
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